Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge prompt code with llm response #458

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

devjpt23
Copy link
Contributor

Problem: Usually, the LLM provides a complete response, but sometimes it returns an incomplete response, often due to an unnecessary comment like "// ... Rest of the code remains unchanged." This "lazy effect" occurs when the LLM avoids generating code that's identical to the input, assuming it can be inferred. This was experienced in #350.

Solution: One option is to re invoke the LLM, prompting it to provide the complete code. However, this approach increases token usage, adding unnecessary cost in the long run. In this PR, we avoid additional invocations by directly merging the code from the prompt with the LLM response wherever the code remains unchanged, achieving the desired result without extra token consumption.

NOTE: This PR is dependent on #436.

@devjpt23 devjpt23 force-pushed the merge-prompt-code-with-llm-response branch from 339f91b to 7c48205 Compare October 27, 2024 19:34
Signed-off-by: devjpt23 <devpatel232408@gmail.com>
@devjpt23 devjpt23 force-pushed the merge-prompt-code-with-llm-response branch 3 times, most recently from 9ba9734 to 1506e69 Compare October 28, 2024 20:41
Signed-off-by: devjpt23 <devpatel232408@gmail.com>
@devjpt23 devjpt23 force-pushed the merge-prompt-code-with-llm-response branch from 1506e69 to 38d6d85 Compare October 28, 2024 21:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant